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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI),  as  part 
of  its  WEBTRAIN  project,  investigated  the  use  of  distributed-learning  technology,  such  as 
immersive  games  and  desktop  simulations.  ARI  seeks  to  provide  the  Army  with  guidance  on 
how  to  develop  instructional  and  motivating  PC-based  training  games. 

The  focus  of  this  research  effort  was  to  identify  characteristics  of  PC-based  games  that 
enhance  instruction  and  motivation,  not  to  assess  a  particular  game.  The  game  used  for  this 
research  was  the  “America’s  Army”  game,  developed  by  the  Office  of  Economic  and  Manpower 
Analysis  located  at  the  United  States  Military  Academy  to  inform  potential  recruits  about  the 
opportunities  available  through  the  U.S.  Army.  The  game  has  been  well  received  by  the  public, 
with  over  two  million  registered  players.  In  October  2002,  the  Assistant  Deputy  Chief  of  Staff 
for  Training,  U.  S.  Army  Training  and  Doctrine  Command  (TRADOC)  agreed  to  sponsor  ARI 
research  to  assess  the  learning  potential  of  the  America's  Army  game.  Data  collection  was 
completed  with  the  assistance  of  the  Advanced  Distributed  Learning  (ADL)  Co-Lab  in 
Alexandria,  VA  and  the  U.S.  Army  recruiting  offices  in  northern  Virginia  (Alexandria, 
Springfield,  and  Falls  Church).  Funding  was  provided  by  ARI. 

The  results  of  this  research  were  briefed  to  the  Director  of  Training  Development  and 
Analysis  Directorate  and  other  TRADOC  representatives  on  3  September  2003  and  to  the 
Assistant  Secretary  of  the  Army  -  Manpower  and  Reserve  Affairs  on  2  October  2003. 


MICHAEL  G.  RUMSEY 


Acting  Technical  Director 
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INSTRUCTIONAL  CHARACTERISTICS  AND  MOTIVATIONAL  FEATURES  OF  A 
PC-BASED  GAME 

EXECUTIVE  SUMMARY 


Research  Requirement: 

PC-based  games  are  currently  being  used  for  instructional  purposes  in  the  U.S.  military, 
but  the  instructional  and  motivational  characteristics  of  such  technology  are  not  well  understood. 
The  requirement  was  to  look  at  a  successful  PC-based  game  to  identify  factors  that  enhanced 
motivation  and  instructional  value. 

Method: 

The  game  used  for  this  research  was  the  “America’s  Army”  game,  a  first-person- 
perspective  game  developed  as  a  recruiting  tool  by  the  Office  of  Economic  and  Manpower 
Analysis  at  the  United  States  Military  Academy.  Twenty  delayed-entry  recruits  and  one  ROTC 
cadet  participated  in  the  research.  Participants  first  completed  a  pre-game  questionnaire  assessing 
prior  knowledge  of  information  presented  in  the  game.  Next,  they  played  the  “basic  training” 
sections  of  the  game,  which  included  Army  background  information,  marksmanship  training,  an 
obstacle  course,  weapons  familiarization,  and  a  MOUT  (military  operations  in  urban  terrain) 
training  mission.  After  playing  the  game,  participants  answered  questions  regarding  information 
presented  during  the  game  and  about  motivational  aspects  of  the  game. 

Findings: 

Participants  recalled  procedures  better  than  facts.  Information  relevant  to  the  progression 
of  the  game  was  recalled  better  than  information  that  did  not  influence  a  player’s  progression. 
Graphic  images  and  spoken  text  were  recalled  more  accurately  than  printed  text.  Motivational 
factors  identified  by  participants  as  influencing  their  likelihood  to  continue  playing  the  game 
were:  challenge  (not  too  hard  and  not  too  easy),  realism  (audio-visual  realism  and  adhering  to 
laws  of  nature),  exploration  (opportunity  to  discover  new  things),  and  control  (manipulating  the 
virtual  environment  through  keyboard/mouse  interface). 

Utilization  of  Findings: 

The  results  of  this  research  provide  useful  information  to  individuals  developing  training 
games,  desktop  simulations,  and  interactive  multimedia  courseware.  The  findings  can  be  used  as 
general  principles  for  developing  training  that  is  both  instructional  and  motivating.  While  the 
research  was  conducted  with  a  PC-based  first-person-perspective  game,  many  of  the  principles 
identified  may  generalize  to  other  types  of  training  games  and  advanced  distributed  learning 
courseware.  A  presentation  based  on  this  research  has  been  incorporated  into  the  TRADOC 
Immersive  Training  Workshop  held  quarterly  for  U.S.  Army  instructors  and  training  personnel. 
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PC-BASED  GAMES/SIMULATIONS: 

INSTRUCTIONAL  CHARACTERISTICS  AND  MOTIVATIONAL  FEATURES 


Introduction 

For  games  to  be  used  for  training  purposes,  they  should  be  motivating,  but  they  also  need 
to  be  instructional.  There  is  no  need  to  develop  a  training  game  that  is  fun  to  play  but  doesn’t 
teach  anything.  The  purpose  of  this  research  was  to  assess  the  instructional  and  motivational 
features  of  a  PC-based  first-person-perspective  game  in  an  effort  to  develop  guidelines  for 
creating  effective  training  games.  In  a  first-person-perspective  game,  the  player  sees  on  the 
display  what  the  character  would  see  in  the  game  environment.  First-person-perspective  games 
tend  to  involve  high-quality  graphics  and  sound  effects  providing  an  immersive  feel  to  players 
because  they  experience  through  the  computer  what  the  game  character  would  experience 
visually  and  auditorily. 

The  America’s  Army  game  was  developed  by  the  Office  of  Economic  and  Manpower 
Analysis  at  the  United  States  Military  Academy  as  a  recruiting  tool  to  inform  the  “recruiting 
age”  public  about  the  U.S.  Army.  The  game  is  a  PC-based  first-person-perspective  game  where 
players  go  through  virtual  “basic  training”  and  then  complete  on-line  military  missions  as  part  of 
a  team.  The  game  has  been  popular;  there  are  over  2  million  registered  players  who  have 
completed  over  a  hundred  million  missions  in  just  over  a  year  (http://www.americasarmy.com). 
Based  on  this  volume  of  usage,  players  certainly  appeared  motivated  to  play  the  America’s  Army 
game.  Therefore,  it  seemed  that  America’s  Army  game  would  be  an  excellent  platform  to 
identify  some  features  that  motivate  users  to  play  first-person-perspective  games.  Also,  given 
that  the  game  was  a  vehicle  to  inform  the  players  about  the  U.S.  Army,  the  characteristics  of  the 
game  that  influenced  learning  could  be  zissessed. 

Training  Games 

The  military  has  used  PC-based  games  for  training  purposes  since  the  1970’s  (Herz  & 
Macedonia,  2002;  Knerr,  Simutis,  &  Johnson,  1979;  Prensky,  2001).  Some  of  these  games  were 
developed  initially  for  the  commercial  purpose  of  entertainment  but  were  modified  for 
instructional  purposes  of  the  military.  An  early  example  was  in  1978,  when  Atari  modified  its 
game  “Battlezone”  for  the  Army  to  use  for  training  tank  tactics  (Prensky,  2001). 

Previous  research  indicates  that  PC-based  games  can  be  effectively  used  for  training 
purposes  (Garris,  Ahlers,  &  Driskell,  2002;  Gopher,  Weil,  &  Bareket,  1994;  Green  &  Bavelier, 
2003;  Knerr  et  al.,  1979;  Prensky,  2001;  Ricci,  Salas,  &  Cannon-Bowers,  1996;  Rieber,  1996; 
Sims  &  Mayer,  2002).  Some  research  demonstrated  that  specific  skills  can  be  trained  with  PC- 
based  games  (Prensky,  2001;  Rieber,  1996;  Sims  &  Mayer,  2002),  while  other  research  showed 
that  training  games  can  teach  generalized  skills  like  troubleshooting  (Knerr  et  al.,  1979)  and 
visual  attention  (Gopher  et  al.,  1994;  Green  &  Bavelier,  2003).  When  comparing  text-based  and 
PC-based  game  instructions,  Ricci  et  al.  found  that  skill  retention  and  satisfaction  with  the 
training  could  be  improved  by  converting  text-based  instruction  to  a  PC-based  game  format.  This 
demonstrated  that  games  can  be  effective  training  tools. 
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While  research  has  indicated  that  PC-based  games  can  be  effective  for  instruction,  it  has 
not  been  clear  which  specific  features  of  games  promoted  learning  or  motivation  to  continue 
using  the  game.  In  a  recent  experiment,  Garris  et  al.  (2002)  used  a  simulation  to  compare  the 
inclusion  of  game  features  such  as  a  high  rate  of  interactivity,  scoring,  and  audio/visual  effects 
against  the  simulation  without  such  features.  They  found  that  the  inclusion  of  all  of  the  game 
features  improved  training  outcomes.  While  Garris  et  al.  identified  a  set  of  game  features  that 
influence  training  effectiveness,  they  did  not  distinguish  between  individual  features.  The  goal  of 
the  current  research  is  to  determine  specific  instructional  features  (how  information  or  concepts 
are  presented  and  integrated  into  the  game)  and  motivational  features  (what  keeps  users  playing 
the  game)  that  influence  the  training  effectiveness  of  PC-based  games. 

Instructional  Aspects 

Although  little  research  has  been  conducted  on  the  specific  components  of  PC-based 
games  that  influence  instructional  effectiveness  (Garris  et  al.,  2002),  there  is  a  broad  base  of 
research  on  interactive  multimedia  available  (Chandler  &  Sweller,  1992;  Clark,  2000;  Craig, 
Gholson,  &  Driscoll,  2002;  Harp  &  Mayer,  1998;  Mayer,  Heiser,  &  Loon,  2001;  Mayer  & 
Moreno,  1998;  Moreno  &  Mayer,  2002;  Mousavi,  Low,  &  Sweller,  1995;  Schraw,  1998;  and 
Tindall-Ford,  Chandler,  &  Sweller,  1997),  Both  PC-based  games  and  interactive  multimedia 
share  common  characteristics,  such  as  audio/visual  presentation  and  user  interaction.  It  is 
reasonable  to  assume  that  results  of  much  of  the  interactive  multimedia  research  may  be 
applicable  to  PC-based  training  games 

For  any  instructional  media,  there  are  many  ways  information  can  be  presented.  In  an 
attempt  to  summarize  findings  in  the  area  of  information  presentation,  Moreno  and  Mayer  (2000) 
described  6  principles  for  producing  greater  learning  of  content  when  using  interactive 
multimedia  instruction:  (a)  split-attention  -  instruction  improves  when  students  are  not  required 
to  split  their  attention  between  multiple  sources  of  similar  information;  (b)  spatial  contiguity  - 
instruction  improves  when  on-screen  text  and  visual  material  are  physically  integrated  rather 
than  separated;  (c)  temporal  contiguity  -  instruction  improves  when  verbal  and  visual  materials 
are  temporally  synchronized  rather  than  asynchronous;  (d)  modality  -  instruction  improves  when 
verbal  information  is  presented  as  speech  rather  than  visually  as  on-screen  text;  (e)  redundancy  — 
the  addition  of  identical  printed  text  during  narration  can  be  distracting  and  decrease  instruction; 
(f)  coherence  -  iiKtruction  improves  when  extraneoiK  material  is  excluded.  Some  of  these 
principles  of  information  presentation  for  interactive  multimedia  instruction  may  apply  to  PC- 
based  training  games. 

As  shown  by  Moreno  and  Mayer  (2000),  how  information  is  presented  can  directly 
impact  the  learning  that  takes  place.  One  objective  of  the  current  research  was  to  evaluate  the 
instructional  value  of  different  forms  of  information  presentation.  In  the  current  research,  we 
assessed  how  information  was  presented  along  three  different  dimensions:  (a)  type  of 
information,  (b)  relevance  to  game  play,  and  (c)  mode  of  presentation. 

Type  of  information.  Tulving  (1972, 1983),  Cohen  and  Squire  (1980),  and  Squire  (1987) 
classified  memories  into  several  different  types.  While  this  work  focused  on  memory,  the  same 
terms  can  be  used  for  how  information  is  presented  in  an  instractional  setting.  For  this  research 
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we  focused  on  three  specific  types  of  information:  procedural,  episodic,  and  factual.  These  three 
types  of  information  map  well  into  Dale’s  (1946)  continuum  of  teaching  methodologies,  which 
includes  doing,  observing,  and  symbolizing.  Doing  involves  direct  experiences  and  simulated 
experiences.  With  doing  there  may  be  interactivity,  while  observing  only  involves  watching 
someone  else  complete  the  task  (live  demonstrations  or  audio-visual  recordings).  When 
observing,  there  is  little  control  by  the  learner  and  minimal  interactivity.  Symbolizing  involves 
instruction  through  text  (printed  and  verbal),  graphs,  and  other  forms  of  symbolic 
communication.  With  symbolizing,  the  actual  skills  and  concepts  to  be  learned  are  not  directly 
shown  but  must  be  derived  from  the  understanding  of  common  semantics. 

For  instructional  purposes,  learners  potentially  acquire  procedural  information  when  they 
actually  perform  a  task  (doing).  Learners  potentially  acquire  episodic  information  when  they 
experience  an  event  as  an  observer  (observing).  Learners  potentially  acquire  factual  information 
when  they  are  provided  semantic-based  (language-based)  information  (symbolizing).  In  a  PC- 
based  first-person-perspective  game  a  player  could  learn  information  conveyed  three  different 
ways:  a)  attempting  the  task  while  receiving  game  feedback  (procedural),  observing  the  game 
environment  (episodic),  or  the  player  could  be  provided  printed  or  spoken  text  (factual).  One  of 
the  objectives  of  the  current  research  was  to  determine  the  relative  training  effectiveness  related 
to  these  three  different  types  of  information  in  the  context  of  a  PC-based  first-person-perspective 
game. 

Relevance  to  game  play.  During  instruction,  some  of  the  information  presented  may  be 
directly  relevant  to  the  instructional  objectives,  while  other  information  may  be  tangential.  The 
inclusion  of  non-relevant  information,  like  jokes  or  non-sequiturs,  can  make  the  instruction  more 
interesting,  but  what  is  the  impact  on  the  training  effectiveness? 

Research  on  multimedia  instruction  has  not  been  favorable  to  adding  irrelevant 
information.  Lower  levels  of  retention  and  transfer  on  the  important  information  occurred  when 
interesting  but  irrelevant  information  was  added  to  multimedia  presentations  (Harp  &  Mayer, 
1997,  1998;  Mayer  et  al.,  2001).  While  these  researchers  illustrated  that  irrelevant  information 
can  distract  learners  viewing  multimedia  presentations,  no  research  has  investigated  the  inclusion 
of  irrelevant  information  in  PC-based  first-person-perspective  games.  One  of  the  objectives  of 
the  current  research  was  to  assess  the  recall  of  information  that  was  relevant  and  was  not  relevant 
to  a  player’s  progression  through  the  game. 

Mode  of  presentation.  In  a  PC-based  first-person-perspective  game  environment,  players 
can  receive  information  through  both  audio  and  visual  modalities.  Also,  visual  information  can 
be  presented  as  text  or  objects.  Training  game  developers  have  a  broad  range  of  options  in  how 
they  present  training  material,  and  one  objective  of  the  current  research  was  to  assess  the  impact 
of  how  the  information  was  presented. 

( 

No  research  to  date  has  assessed  the  effect  of  presentation  mode  on  information  recall  in 
the  context  of  PC-based  games.  Furthermore,  research  in  multimedia  instruction  seems  to  be 
mixed.  Tindall-Ford  et  al.  (1997)  and  Leahy,  Chandler,  and  Sweller  (2003)  found  that  when 
audio  and  visual  information  are  complementary  to  one  another  they  are  more  effective  than 
either  one  by  itself  Mayer  and  Moreno  (1998)  found  that  students  retained  identical  text  from 


3 


multimedia  instructional  presentations  better  through  audio  channels  than  through  visual 
channels.  Conversely,  in  some  situations,  printed  text  may  be  superior  to  audio  text.  Eberman 
and  McKelvie  (2002)  found  that  when  asked  to  visualize  the  images  related  to  the  text,  the 
content  of  printed  text  was  recalled  better  than  audio  information.  These  differences  may  be  due 
to  the  training  environment  used.  The  current  research  further  investigated  which  modes  of 
presentation  improve  learning  in  the  specific  learning  environment  of  a  first-person-perspective 
training  game. 

Motivational  Aspects 

Motivation  may  be  defined  as  what  drives  purposeful  behavior  (Lawler,  1994).  With 
regards  to  instruction,  the  purposeful  behavior  can  be  interaction  with  the  instructional  media;  in 
the  case  of  this  research,  playing  the  game.  For  the  current  research,  the  objective  was  to  identify 
features  of  a  firet-person-perspective  game  that  motivate  a  user  to  continue  playing  the  game.  An 
understanding  of  motivating  features  may  allow  training  game  designers  to  keep  learners 
engaged  in  the  training  activity  for  extended  periods. 

Training  motivated  learners  is  far  easier  than  training  non-motivated  learners  (Prensky, 
2001).  Sometimes  the  subject  matter  provides  the  motivation  for  the  learner,  while  other  times 
how  the  instruction  is  conducted  can  influence  motivation.  The  theory  behind  training  games 
suggests  that  the  act  of  playing  a  game  should  motivate  the  learners  to  continue  playing,  thereby 
continuing  to  learn.  If  information  is  embedded  in  the  game,  than  continuance  in  playing  the 
game  should  result  in  more  material  being  introduced  to  the  learner. 

In  a  series  of  experiments,  Malone  (1981)  and  Malone  and  Lepper  (1987)  identified  four 
features  of  PC-based  games  that  promote  motivation  to  continue  playing.  The  four  features  were: 
(^)  challenge  -  activity  provides  an  intermediate  level  of  difficulty;  (b)  control  ~  ability  to 
determine  the  outcome  of  events  based  on  the  player’s  actions;  (c)  curiosity  -  belief  that  the 
player  will  uncover  something  new;  and  (d)  fantasy '  feeling  that  players  Me  engaging  in  an 

activity  that  is  not  real.  In  Edition  to  Malone’s  research  on  motivation  for  games,  ofiiers  have 
also  assessed  features  that  influence  motivation.  For  example,  Corbeil  (1999)  investigated  role- 
playing  and  first-person-perspective  games  and  identified  six  motivating  aspects:  (a)  self- 
discovery,  constructivism;  (b)  self-validation,  efficacy;  (c)  escape,  a  break  from  reality;  (d) 
camaraderie,  social;  (e)  creative  competition,  feedback;  and  (f)  a  metaphor  for  life,  related  to 
reality.  Garris  et  al.,  (2002)  reviewed  previous  research  on  motivation  in  training  games  and 
found  six  similar  game  characteristics  which  they  thought  would  influence  motivation:  fantasy, 
rules/goals,  sensory  stimuli,  challenge,  mystery,  and  control.  In  the  current  research,  we 
identified  characteristics  of  a  particular  first-person-perspective  PC-based  game  that  may 
motivate  players  to  continue  playing  and  characteristics  that  may  not  motivate  continued  play. 

Method 

Participants 

Twenty-one  individxials  participated  in  the  experiment.  The  average  age  of  the 
participants  was  20.4  years  old,  with  a  range  from  17  to  29.  Twenty  of  the  participants  were 
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delayed  entry  recruits,  i.e.,  individuals  who  made  a  commitment  to  enlist  but  had  not  yet  started 
basic  training.  One  participant  was  a  first-year  member  of  an  ROTC  (Reserve  Officers  Training 
Corps)  program.  While  all  of  the  participants  had  made  some  commitment  to  the  U.S.  Army, 
none  had  gone  through  basic  training. 

Apparatus 

The  America’s  Army  game  was  used  as  the  platform  for  this  experiment.  The  America’s 
Army  game  is  a  first-person-perspective  adventure/shooting  game  with  advanced  graphics  and 
sound  effects.  In  this  game,  players  go  through  virtual  “basic  training”  and  then  complete  on-line 
military.missions  as  part  of  a  team.  The  minimum  system  requirements  for  America’s  Army 
include:  Microsoft  Windows  98/ME/XP/2000,  a  Pentium  III  processor  running  at  700  MHz  with 
128  MB  of  RAM,  1.5  GB  of  hard  drive  space,  and  an  internet  connection.  This  research  focused 
on  the  individual  player  “basic  training”  section. 

Procedure 

Pre-game  data  collection.  Participants  were  first  asked  to  complete  demographic 
questions  and  a  nine-item  multiple-choice  questionnaire  to  measure  prior  knowledge  about  the 
game  and  the  U.S.  Army.  Each  item  had  five  probable  answers  including  an  “I  don’t  know” 
choice.  See  Appendix  A  for  the  pre-game  questionnaire. 

Play  game.  After  completing  the  pre-game  questioimaire,  participants  then  played  the 
first  four  sections  of  the  game:  a)  marksmanship  training,  b)  an  obstacle  course,  c)  weapons 
familiarization,  and  d)  a  MOUT  (military  operations  in  urban  terrain)  training  mission. 

The  marksmanship  section  began  with  a  printed  text  description  of  basic  training, 
marksmanship  training,  and  a  brief  history  of  Fort  Beiming.  Next,  a  computer-generated  drill 
instractor  verbally  explained  what  was  required  in  the  marksmanship  training  section.  The  player 
then  took  a  position  in  the  shooting  range  with  their  rifle  (M-16A2)  and  ammunition.  The  player 
was  provided  instructions  on  tasks  like  loading  the  gun  and  the  number  of  targets  they  needed  to 
hit  to  qualify  with  their  weapon.  The  practice  and  qualification  roimds  were  repeated  imtil  the 
participant  qualified  (at  least  23  out  of  40  targets  with  40  rounds). 

The  next  section  was  the  obstacle  course,  which  began  with  the  same  printed  text  pages 
about  basic  training  and  the  history  of  Fort  Benning.  Additional  text  was  added  to  describe 
obstacle  course  training  in  the  U.S.  Army.  Following  the  text  sections,  instructions  for  player 
movement  were  given  verbally  by  the  drill  instructor  while  the  player  virtually  ran  through  the 
course  on  a  practice  run.  The  obstacle  course  included  obstacles  like  climbing  over  a  wall, 
miming  over  a  balance  beam,  and  low  crawling  under  barbwire.  After  completing  the  practice 
mn,  the  player  ran  the  entire  course  against  a  clock.  Each  player  repeated  this  until  they  bettered 
the  time  requirement  of  90  seconds. 

The  third  section  was  weapons  familiarization,  which  began  with  printed  text  about  basic 
training,  the  history  of  Fort  Benning,  and  a  description  of  some  of  the  weapons  used  by  the  U.S. 
Army.  Then,  the  players  had  the  opportunity  to  use  four  different  weapons:  a  machine  gun,  a 
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rifle  with  a  grenade  launcher,  fragmentation  grenades,  and  smoke  grenades.  Unlike  in  the 
previous  sections,  there  was  no  standard  to  meet  before  going  on  to  the  next  section. 

The  final  section  was  MOUT  training,  which  began  with  another  printed  text  presentation 
about  basic  training,  the  history  of  Fort  Benning,  and  a  description  of  infantry  training.  Then, 
players  had  to  navigate  through  a  building  and  various  tunnels  while  shooting  at  stationary 
silhouettes  of  combatants,  distinguishing  them  from  the  silhouettes  of  innocent  bystanders. 
During  this  section  the  players  used  movement  skills  developed  in  the  obstacle  course  and  their 
shooting  skills  developed  in  marksmanship  and  weapons  familiarization.  This  section  also 
introduced  “rules  of  engagement”,  which  dictated  that  one  was  to  only  shoot  “hostile”  targets, 
while  not  shooting  “noncombatant”  targets. 

Post-game  data  collection.  Upon  completion  of  “Basic  Training”  each  participant 
completed  a  35-item  multiple-choice  questionnaire  on  information  presented  during  the  game 
(Appendix  B).  As  with  the  pre-game  questionnaire,  each  multiple-choice  question  had  five 
possible  answers,  including  an  “I  don’t  know”  choice.  Nine  of  the  multiple-choice  questions 
were  repeated  from  the  pre-game  data  collection,  so  the  remaining  26  were  used  to  assess 
instructional  features.  Also  included  in  the  post-game  questionnaire  were  five  open-ended 
questions,  four  of  which  were  used  to  assess  game  characteristics  that  might  influence 
motivation.  The  first  open-ended  question  was  “what  about  the  ‘America’s  Army’  game  would 
make  you  want  to  play  file  game  again?”  This  question  directly  measured  what  features  of  the 
game  motivated  participants  to  want  to  continue  interaction  with  the  program.  The  second 
question,  “which  section  of  the  basic  combat  training  did  you  like  the  most,  and  why?’  prompted 
responses  regarding  what  they  like  most  about  the  game.  It  was  thought  that  assessing  the 
positive  features  of  the  game  would  indicate  reasons  participants  would  be  motivated  to  continue 
playing  the  game  (motivating  features).  The  third  and  fifth  questions  assessed  what  about  the 
game  they  liked  least  or  would  change,  which  should  reflect  non-motivating  characteristics  of  the 
game.  The  fourth  question  was  a  motivation-neutral  question  assessing  what  they  learned  during 
the  game  and  was  not  used  in  this  report. 

The  26  multiple-choice  questions  that  appeared  only  in  the  post-game  questioimaire  were 
classified  along  three  different  instructional  characteristics:  type  of  information,  relevance  of 
information  to  game  play,  and  mode  of  presentation.  Items  were  classified  as  belonging  to 
different  subsets  of  information  type  using  the  following  definitions:  (a)  procedural  —  cognitive 
or  motor  skills  and  activities;  (b)  episodic  -  experiential  memories  of  sensation,  perception,  and 
past  events;  and  (c)  factual  -  facts  and  concepts  represented  by  text  and  symbols.  For  relevance 
to  game  play,  items  were  classified  as  belonging  to  the  subsets  defined  as:  (a)  relevant  - 
information  that  is  required  or  helpful  to  progress  in  the  game  and  (b)  irrelevant  —  information 
that  does  not  impact  on  progress  in  the  game.  Finally,  items  were  classified  by  their  mode  of 
presentation  using  the  following  subset  definitions:  (a)  spoken  text  -  narrated  information,  (b) 
printed  text  -  printed  information,  and  (c)  graphic  images.  Five  questions  on  the  post-game 
questionnaire  assessed  information  that  was  presented  through  more  than  one  modality,  so  only 
21  questions  were  used  in  the  assessment  of  the  effect  of  presentation  modality.  The 
classifications  (p=procedural,  e=episodic,  f=factual,  rl=relevant,  ir=irrelevant,  spk=spoken  text, 
pm=printed  text,  and  grp=graphic  image)  for  each  question  are  presented  next  to  the  question  in 
Appendix  B. 
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After  these  categories  and  their  definitions  were  developed/researched,  two  of  the 
researchers  separately  classified  each  of  the  items  into  their  appropriate  information  type, 
relevance,  and  presentation  mode  categories.  The  initial  inter-rater  agreement 
[agreements/(agreements  +  disagreements)]  for  the  characteristics  of  information  type,  relevance 
to  game  play,  and  mode  of  presentation  were  deemed  acceptable  at  .91,  .89,  and  1.0, 
respectively.  The  items  that  authors  disagreed  on  initially  were  then  discussed  until  a  mutual 
consensus  for  categorization  was  reached. 

Responses  were  grouped  into  four  categories  based  on  previous  research  on  motivation 
(Malone,  1981;  Malone  &  Lepper,  1987)  and  the  terms  used  by  the  participants  in  their 
responses.  These  categories  were;  challenge,  control,  realism,  and  exploration.  Challenge  was 
defined  as  a  response  that  discussed  accomplishing  the  tasks  required  to  continue  on  to  the  next 
section  in  the  game  or  goal  achievement  (e.g.,  “it  was  fun  trying  to  complete  the  obstacle  course 
in  only  90  seconds”).  Control  was  defined  as  a  response  regarding  the  interaction  of  the  player 
with  the  game  environment  (e.g.,  “I  enjoyed  seeing  the  targets  fall  when  I  shot  them”).  Realism 
was  defined  as  responses  about  elements  that  made  the  game  experience  more  representative  of  a 
real-life  experience.  This  category  included  comments  about  the  games  high  visual  and  audio 
fidelity,  as  well  as  responses  about  realistic  weapons  and  procedures.  Exploration  was  defined  as 
responses  that  discussed  the  process  of  discovery  and  novel  sensory  stimulation  (e.g.,  seeing  a 
new  weapon  or  participating  in  a  new  activity  like  MOUT). 

Two  researchers  independently  coded  all  responses  to  the  open-ended  questions.  The 
codings  of  both  researchers  were  compared  to  calculate  inter-rater  agreement.  Inter-rater 
agreement  for  questions  about  what  motivated  participants  to  continue  playing  and  the  initial 
inter-rater  agreement  for  questions  about  what  failed  to  motivate  were  deemed  acceptable  at  .81 
and  .87,  respectively.  The  responses  that  coders  initially  disagreed  on  were  then  discussed  rmtil 
consensus  was  reached. 

Results 


Pre-game  data 

The  nine  pre-game  questionnaire  items  were  used  to  determine  whether  participants  were 
naive  to  the  information  presented  during  the  game.  The  mean  score  for  the  pre-game 
questionnaire  was  19.6%  with  a  standard  deviation  of  16.4%  (range  =  0-56%),  indicating  that 
participants  were  not  familiar  with  much  of  the  game  related  information. 

Information  Type 

Three  different  types  of  information  were  assessed  through  the  post-game  questionnaire: 
procedural  (knowledge  of  a  skill  or  activity);  episodic  (knowledge  of  past  events,  experiential 
memories  of  sensation  and  perception);  and  factual  (knowledge  that  one  represents  symbolically, 
such  as  facts  and  concepts).  Of  the  26  items  used  to  assess  information  type,  6  were  classified  as 
procedural,  8  as  episodic,  and  12  as  factual. 
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The  percentage  of  questions  answered  correctly  for  each  subset  of  information  type  are 
displayed  in  Figure  1 .  The  I-bars  indicate  the  standard  error  for  each  type  of  information  subset. 
For  procedural  information,  across  all  participants  the  mean  percentage  correct  was  77.9%  with  a 
standard  error  of  5.3%.  For  episodic  information,  the  mean  was  70.5%  with  a  standard  error  of 
3.1%.  For  factual  information,  the  mean  was  62.9%  with  a  standard  error  of  4.3%.  The 
difference  betvi^een  the  mean  accuracy  scores  was  significantly  different  (one-way  ANOVA, 
F=AS),p=  .026).  A  Tukey  pairwise  comparison  found  a  significant  difference  between  the 
procedural  and  factual  means  (p<  .05).  Using  Cohen’s  d,  the  effect  size  between  procedural  and 
factual  was  .73.  The  effect  size  between  episodic  and  factual  was  .37,  and  the  effect  size  between 
procedural  and  episodic  was  .36.  Cohen’s  d  was  calculated  as  [Cohen's  d  =  (Mean/  -  Mean2)  / 
standard  deviation  pooled]- 
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Figure  1.  The  mean  percentage  correct  for  questions  involving  the  three  different  types  of 

information  (procedural,  episodic,  and  factual). 

Relevance  to  Game  Play 

Questions  from  the  post-game  questionnaire  were  categorized  as  either  relevant  to  player 
progress  through  the  game  or  irrelevant  to  game  play.  Of  the  26  items  pertinent  to  the  issue  of 
relevance,  16  were  classified  as  relevant  to  game  progression,  while  the  remaining  10  were 
classified  as  irrelevant. 

The  percentage  of  questions  answered  correctly  for  each  relevance  category  is  displayed 
in  Figure  2.  For  relevant  information,  the  mean  accuracy  score  was  72.3%  with  a  standard  error 
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of  3.6%.  For  irrelevant  information,  the  mean  was  58.7%  with  a  standard  error  of  4.8%.  Relevant 
information  was  recalled  at  a  statistically  significant  higher  level  than  irrelevant  information 
(paired  sample  t-test,  f=2.29,  jp>.01).  Using  Cohen’s  d,  the  effect  size  between  relevant  and 
irrelevant  was  .65. 
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Figure  2.  The  mean  percentage  correct  for  questions  involving  relevant  and  irrelevant 
information. 

Presentation  Modality 

Questions  from  the  post-game  questioimaire  were  categorized  based  on  presentation 
modality  (printed  text,  spoken  text,  and  graphic  images).  Of  the  21  questions  used  to  assess 
presentation  modality,  6  were  classified  as  graphic  images,  4  as  spoken  text,  and  1 1  as  printed 
text.  Five  questions  were  not  used  in  the  analysis  of  this  variable,  because  the  information  was 
presented  in  more  than  one  modality. 

The  percentage  of  questions  answered  correctly  for  each  category  of  presentation 
modality  is  displayed  in  Figure  3.  For  graphic  images,  the  mean  was  79.1  %  with  a  standard  error 
of  3.4%.  For  spoken  text,  the  mean  was  73.8%  with  a  standard  error  of  5.0%.  For  printed  text, 
the  mean  was  57.1%  with  a  standard  error  of  4.3%.  The  difference  between  the  mean  scores 
based  on  modality  of  presentation  was  significantly  different  (one-way  ANOVA,  F=9.3,p<.01). 
A  Tukey’s  pairvv'ise  comparison  identified  a  significant  difference  between  graphic  images  and 
printed  text  (p<.01)  and  a  difference  between  spoken  text  and  printed  text  (p<.05).  Using 
Cohen’s  d,  the  effect  size  of  the  difference  between  graphics  and  printed  text  was  1 .02,  while  the 
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effect  size  for  spoken  text  verses  printed  text  was  .77.  The  effect  size  for  the  difference  betw'een 
graphics  and  spoken  text  was  .25.  Cohen’s  d  was  calculated  as  [Cohen's  d  =  {Mean\  -  Meani)  / 
standard  deviation  pooled]- 


Figure  3.  The  mean  percentage  correct  for  questions  involving  the  three  different  types 


presentation  formats  of  information. 


Motivation 


Features  that  motivate.  There  were  25  responses  to  the  two  open-ended  questions 
assessing  the  motivational  features  of  the  game.  Not  all  participants  responded  to  the  questions 
used  to  assess  motivation,  and  some  participants  wrote  more  than  one  response.  The  responses 
for  what  might  motivate  them  to  continue  playing  the  game  fell  within  four  categories:  realism, 
challenge,  exploration,  and  control  (see  Figure  4).  Twenty  of  the  responses  were  coded  into  one 
of  the  four  categories;  nine  mentioned  realism,  six  mentioned  challenge,  three  mentioned 
exploration,  and  two  mentioned  control  over  the  game  environment.  Additionally,  there  were 
five  responses  that  were  not  expressed  in  the  figure  because  the  responses  did  not  clearly  express 
a  reason  why  the  game  was  motivating;  they  were  either  tautological  (i.e.,  “I  liked  it  because  it 
was  fim”)  or  statements  that  they  found  no  part  of  the  game  motivating. 


10 


Figure  4.  Proportion  of  responses  by  category  of  feature  that  motivated  participants  to  continue 
playing. 

Features  that  did  not  motivate.  There  were  25  responses  to  the  two  open-ended  questions 
assessing  the  non-motivational  aspects  of  the  game.  These  fell  within  the  3  categories  of  realism, 
challenge,  and  control  (Figure  5).  Eighteen  of  the  responses  were  coded  into  one  of  the  following 
categories,  eight  responses  mentioned  a  lack  of  control,  five  mentioned  the  game  not  being 
challenging  enough,  three  stated  that  the  game  was  too  challenging,  and  two  mentioned  a  lack  of 
realism.  Seven  responses  did  not  clearly  express  reasons  the  game  was  not  motivating  and  were 
not  expressed  in  the  figure. 
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Figure  5.  Proportion  of  responses  describing  features  that  did  not  motivate  participants  to 
continue  playing. 

Discussion 

Instructional  Characteristics 

Information  type.  In  the  current  experiment,  subjects  recalled  procedural  information  at  a 
higher  rate  than  factual  information.  This  indicates  that  within  the  context  of  PC-based  first- 
person-perspective  games,  procedural  information  was  learned  better  than  factual  information. 
This  finding  relates  to  Dale’s  (1946)  continuum  of  instruction  methodology  of  doing,  observing, 
and  symbolizing.  Doing  correlates  with  procedural  information,  observing  correlates  with 
episodic  information,  and  symbolizing  correlates  with  factual  information.  The  current  research 
supports  Dale’s  guidelines  that  what  is  done  (procedural)  is  learned  best,  followed  by  what  is 
observed  (episodic),  and  symbolic  information  (factual)  is  least  likely  to  be  learned. 

While  participants  performed  better  on  procedural  and  episodic  questions,  factual 
information  was  still  recalled.  Some  instructional  information  will  be  naturally  structured  as 
factual  information;  therefore,  factual  information  should  not  be  disregarded  completely  but 
should  be  used  in  a  targeted  manner. 

Little  research  has  compared  the  retention  of  different  types  of  information  presented  in 
training  environments.  In  research  on  individuals  with  amnesia,  they  were  more  likely  to 
remember  procedural  information  than  factual  information  (Cohen  &  Squire,  1980;  Squire, 
1987).  While  these  findings  are  in  agreement  with  the  current  research  findings  and  may  suggest 
that  procedural  information  is  more  resistant  to  forgetting  than  factual  information,  additional 


research  with  training  games  is  needed  to  demonstrate  the  generality  of  the  current  research 
results. 


Relevance.  In  the  current  experiment,  information  that  was  relevant  to  the  progression  of 
the  game  was  recalled  more  accurately  than  information  tangential  to  player  progression.  This 
suggests  that  training  game  developers  should  incorporate  learning  objectives  into  the  storyline 
of  the  game.  If  the  training  objectives  are  not  part  of  the  game’s  storyline,  the  player  may 
remember  how  to  play  the  game  instead  of  learning  the  training  objectives. 

Beyond  the  beneficial  effect  that  relevant  information  may  have,  there  may  also  be  a 
damaging  effect  of  having  information  that  is  not  relevant  to  the  topic  of  instruction.  As 
mentioned  in  the  introduction,  it  has  been  demonstrated  in  multimedia  instruction  research  that 
the  inclusion  of  non-relevant  details  can  be  distracting  and  have  detrimental  effects  on  retention 
and  learning  transfer  (Harp  &  Mayer,  1997, 1998;  Mayer  et  ah,  2001).  This  line  of  research, 
along  with  the  findings  of  the  current  experiment,  suggest  that  relevant  information  will  be 
recalled  better  than  irrelevant  information,  and  the  inclusion  of  irrelevant  information  is  less 
likely  to  be  recalled  and  in  some  cases  may  possibly  impede  the  learning  of  the  relevant 
information. 

Mode  of  presentation.  Findings  indicate  that  spoken  text  and  graphic  images  were 
recalled  more  accurately  than  printed  text.  This  extends  previous  findings  of  Mayer  and  Moreno 
(1998)  to  a  first-person-perspective  game,  where  they  found  that  students  retained  identical  text 
from  multimedia  instructional  presentations  better  through  audio  channels  than  through  visual 
channels.  This  suggests  that  across  a  range  of  audio-visual  environments,  information  presented 
as  spoken  text  may  lead  to  better  retention  than  printed  text. 

Given  our  findings,  one  may  think  that  only  graphic  images  or  spoken  text,  and  not 
written  text,  should  be  used  for  instruction  in  this  medium.  However,  for  several  reasons,  this  is 
not  the  case.  First,  a  casual  observation  during  the  experiment  indicated  that  some  participants 
skimmed  or  ignored  full  pages  of  text  by  merely  clicking  on  “next”  to  proceed  to  the  next  page 
without  ample  reading  time.  This  suggests  that  large  blocks  of  written  text  may  be  ignored  by 
some  players  in  first-person-perspective  games  and  may  account  for  these  findings.  It  is 
important  to  note  that  text  was  recalled,  but  only  to  a  lesser  extent.  This  may  simply  be  due  to  the 
fact  that  students  were  able  to  control  whether  or  not  they  could  skip  the  written  text  information. 
This  was  not  an  option  for  spoken  text  or  graphic  images.  Differences  may  be  due  to  whether  or 
not  participants  received  the  information  rather  than  how  it  was  displayed.  Future  research 
should  investigate  this  difference.  Secondly,  previous  research  has  shown  that  when  audio  and 
visual  information  are  complementary  to  one  another  they  are  more  effective  than  either  one  by 
itself  (Leahy  et  al.,  2003;  Tindall-Ford  et  al.,  1997)  Therefore,  the  combination  of  text  and 
graphic  images,  whereby  one  complements  the  other,  may  be  an  effective  way  to  provide 
instruction. 

Motivation  Features 

Based  on  the  responses  of  the  participants  to  the  open-ended  questions,  there  were  four 
game  features  that  influenced  motivation  to  continue  playing:  challenge,  realism,  control,  and 
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exploration.  These  four  categories  are  similar  to  those  presented  by  Malone  (1981)  and  Malone 
and  Lepper  (1 987),  which  were:  challenge,  fantasy,  control  and  curiosity. 

Challenge.  Across  features  that  positively  and  negatively  affected  motivation,  challenge 
was  the  most  frequently  mentioned.  This  indicates  the  potential  importance  of  attaining  an 
optimal  level  of  difficulty  so  as  to  motivate  participants,  a  finding  that  is  in  line  with  Malone’s 
research  on  motivational  features  of  PC-based  games  (Malone,  1981;  Malone  &  Lepper,  1987). 
Malone  stated  that  an  optimal  level  is  reached  when  a  participant  finds  an  activity  difficult, 
without  finding  it  overwhelming.  If  the  task  is  too  simple  and  the  person  is  guaranteed  success,  it 
may  be  perceived  as  boring,  and  if  it  is  too  difficult,  the  individual  may  become  fmstrated 
(Rieber,  Davis,  Mat2ko,  &  Grant,  2001). 

Individuals  in  the  current  research  reported  sections  of  the  game  as  being  too  challenging 
or  not  challenging  enough.  It  is  likely  that  participants  came  in  with  different  amounts  of  game 
playing  experience;  therefore,  it  is  not  surprising  to  find  that  a  specific  level  of  difficulty  was  not 
applicable  across  all  participants.  In  order  to  challenge  an  individual  at  an  appropriate  level, 
training  games  can  be  designed  with  a  dynamic  level  of  difficulty  (Garris  et  al.,  2002;  Malone  & 
Lepper,  1987;  Prensky,  2001).  As  a  player  progresses  through  the  game  easily  the  level  of 
difficulty  increases,  or  if  a  player  has  problems  the  level  of  difficulty  decreases.  A  training  game 
where  the  degree  of  challenge  matches  die  performance  level  of  the  player  may  motivate  the 
broadest  range  of  players. 

Control.  Control  existed  when  a  player  felt  that  their  performance  was  directly  related  to 
outcomes  in  the  game  environment.  In  this  experiment,  only  a  few  responses  mentioned  control 
as  a  motivating  feature  (2),  while  a  larger  number  of  responses  (8)  identified  a  lack-of-control  as 
being  a  negative  feature  and  potentially  non-motivational.  The  prevalence  of  negative  remarks 
relative  to  positive  remarks  suggests  that  either  a  sense  of  control  was  more  fi-equently  absent 
than  present  in  the  game  or  that  subjects  have  a  tendency  to  notice  the  absence  of  control  more 
dien  its  presence.  In  eitiier  case,  the  data  suggest  that  training  game  designers  should  provide 
players  with  a  sense  of  control  over  the  game  environment. 

Malone  and  Lepper  (1987)  suggested  that  controlling  one’s  environment  is  a  motivating 
factor  contributing  to  enjoyment  of  PC-based  games.  Previous  research  found  a  relationship 
between  completion  rates  in  academic  online  courses  and  feelings  of  control.  Students  who  felt 
more  in  control  of  their  environment  had  higher  completion  rates  of  courses  than  individuals  who 
did  not  feel  in  control  (Dille  &  Mezack,  1991 ;  Parker,  2003).  Therefore,  it  is  likely  that 
individuals  may  be  more  willing  to  persist  in  a  training  game  if  it  endows  them  with  feelings  of 
control. 

Greater  control  has  also  been  linked  with  increased  learning.  Research  has  shown  that 
when  an  individual  controls  the  rotation  of  a  three-dimensional  object  they  have  faster 
recognition  of  the  object  than  individuals  who  observed  the  same  rotation  passively  (James, 
Humphery,  Vilis,  Corrie,  Baddour,  &  Goodale,  2002).  These  findings  along  with  the  findings  of 
Parker  (2003)  and  Dille  and  Mezack  (1991)  suggest  that  a  sense  of  control  leads  to  both  higher 
motivation  and  greater  learning. 
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Realism.  Realism  was  the  most  commonly  mentioned  reason  to  continue  playing  the 
game.  Some  participants  also  mentioned  that  a  lack  of  realism  was  a  reason  that  they  did  not  find 
parts  of  the  game  motivating,  suggesting  that  realism  is  an  important  feature  of  first-person- 
perspective  games. 

Malone’s  (1981)  element  of  fantasy  relates  to  the  category  of  realism  used  in  the  current 
research.  It  could  be  argued  that  the  realistic  nature  of  the  game  promoted  the  fantasy 
(immersion)  that  individuals  were  engaged  in  something  beyond  just  playing  a  game.  The 
inclusion  of  fantasy,  or  “suspension  of  disbelief’,  could  be  motivational  in  a  training  game 
because  it  provides  a  context  for  the  learning  activity  that  provides  experiences  not  otherwise 
readily  available  to  an  individual  (e.g.,  qualifying  with  a  rifle,  throwing  a  grenade,  or  completing 
an  obstacle  course),  all  while  sitting  in  front  of  a  computer. 

Outside  of  motivational  benefits,  there  is  an  additional  purpose  to  creating  a  realistic 
environment.  It  has  been  demonstrated  that  increased  transfer  of  training  occurs  when  there  was 
greater  similarity  between  the  training  environment  and  the  performance  environment  (Aufffey, 
Mirabella,  &  Siebold,  2001;  Detterman,  1993;  Thorndike  &  Woodworth,  1901).  Given  that  the 
purpose  of  training  games  are  to  improve  performance  on  an  actual  task,  the  games  should 
realistically  represent  the  elements  of  the  real-world  environment  that  are  pertinent  to  the 
instructional  objectives  of  the  training  game. 

While  realism  may  be  beneficial,  training  game  developers  should  be  wary  of  overly 
investing  resources  in  graphics  and  audio  realism.  High  fidelity  graphics  and  audio  can  be 
expensive  to  produce,  and  it  appears  that  there  are  diminishing  educational  returns  from  high 
fidelity  systems  (Baum,  Riedel,  Hays,  &  Mirabella,  1982;  Freda  &  Ozkaptan,  1980;  Morris  & 
Tarr,  2002).  For  example,  Morris  and  Tarr  (2002)  found  that  quality  of  decision-making  actually 
decreased  with  higher  levels  of  graphics.  This  suggests  that  while  realism  is  a  feature  that  may 
motivate  players,  there  is  a  limit  where  greater  levels  of  realism  may  no  longer  increase  the 
effectiveness  of  a  training  game. 

Exploration.  Proper  use  of  exploration  was  mentioned  as  motivational,  but  was  not 
mentioned  as  un-motivational.  There  are  two  possible  explanations  for  this  finding.  First,  this 
particular  game  could  have  stimulated  adequate  levels  of  exploration  for  all  individuals.  The 
second  possible  reason  is  that  individuals  might  not  readily  notice  inadequate  levels  of 
exploration;  therefore  they  would  not  report  the  absence  of  exploration.  Whatever  the  reason, 
future  research  should  investigate  the  role  of  exploration  in  learner  motivation. 

The  feature  of  exploration  is  similar  to  Malone’s  (1981)  curiosity.  Malone  considers 
curiosity  to  occur  when  an  individual  searches  out  knowledge  gaps.  Researchers  have  suggested 
that  curiosity  is  a  fundamental  and  powerful  human  drive  to  acquire  novel  information,  for  no 
immediate  external  purpose  and  thus  for  no  immediate  external  reward  (Kashdan  &  Fincham, 
2002).  Thus,  this  particular  motivational  element  in  a  PC-based  game  is  naturally  entwined  with 
the  learning  process  and  may  be  harnessed  for  instructional  purposes. 

Optimal  levels  of  game  features.  Of  the  four  motivational  features  mentioned  as  reasons 
for  continuing  to  play  the  game,  three  of  those  features  were  also  mentioned  as  reasons  for  not 
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continuing  to  play  the  game.  This  indicates  that  not  all  of  the  four  motivational  features  were  at 
optimal  levels.  For  example,  with  regards  to  challenge,  some  individuals  identified  sections  of 
the  game  as  too  difficult,  while  others  identified  sections  as  too  easy.  In  the  case  of  realism, 
many  participants  praised  the  level  of  the  game’s  realism,  while  a  few  individuals  identified  the 
game  as  not  realistic  enough.  An  attempt  to  identify  a  single  optimal  level  of  any  one  of  these 
features  would  be  difficult  because  preference  varies  across  individuals.  Differences  in 
individual  preference  for  control,  realism,  exploration  and  challenge  could  originate  from  a 
myriad  of  sources,  such  as  culture,  current  mood,  hand-eye  coordination,  past  play  experiences 
and  visual  preferences.  Because  of  individual  differences,  a  system  that  allows  for  the  variation 
of  these  levels  may  be  beneficial  to  learner  motivation. 

Many  variable  systems  have  been  used  in  commercial  games.  Most  games  offer  variable 
levels  of  difficulty,  where  a  player  may  select  between  beginner,  intermediate,  and  expert  levels. 
Some  games  allow  for  varying  levels  of  realism  by  permitting  a  user  to  select  the  level  of  “gore”, 
which  determines  how  much  blood  will  spurt  when  a  person  is  shot.  Other  games  contain 
optional  “quests”  for  individuals  who  wish  to  engage  in  more  or  less  exploration  of  the  game. 
Additionally,  there  are  games  that  allow  users  to  choose  a  level  of  assistance  for  help  controlling 
the  game,  where  they  can  toggle  “auto-aim”  and  the  computer  controls  the  aiming  of  the  player’s 
gun.  Varying  these  features  allows  the  game  to  appeal  to  a  broader  range  of  players. 

The  control  of  these  systems  may  take  many  forms.  One  option  is  to  allow  the  player  to 
set  the  level  of  the  particular  game  feature.  A  second  option  is  where  an  instructor  would  select 
the  level  based  on  the  instructional  objectives.  A  third  mechanism  would  be  to  have  an 
automated  system  that  attempts  to  identify  the  optimal  level  for  a  player  based  on  their 
performance.  An  early  version  of  this  mechanism  of  variable  levels  of  challenge  was  used  in  the 
arcade  game  “Pac-Man”  (Bowman,  1982),  where  players  began  at  a  simple  level  and  progressed 
to  harder  levels  when  they  were  successful  at  a  prior  level.  A  combination  of  these  features  may 
be  appropriate.  For  example,  the  game  designer/instructor  might  select  the  level  of  realism  based 
on  the  training  objectives,  the  player  might  select  the  level  of  exploration  and  control,  while  the 
game  might  regulate  the  level  of  challenge  based  on  the  player’s  performance. 

In  this  research,  all  four  motivational  features  were  mentioned  as  reasons  players  may  be 
motivated  to  continue  playing  the  game,  while  only  three  of  the  categories  were  mentioned  as 
reasons  that  may  be  non-motivating  with  regards  to  players  continuing  to  play  the  game 
(challenge,  control,  and  realism).  However,  it  is  not  clear  in  these  data  as  to  what  mix  of  these 
features  are  most  beneficial  towards  motivating  participants  to  continue  playing.  Nor  do  we  have 
data  that  support  the  actual  continuance  of  playing.  These  data  simply  provide  participants 
reactions  on  what  they  found  most  interesting  and  engaging,  which  should  indicate  what  factore 
would  lead  them  to  play  in  the  future.  Previous  research  by  Malone  and  Lepper  (1987)  noted  that 
while  these  elements  influence  motivation,  their  presence  will  not  guarantee  motivation  nor  will 
their  absence  preclude  motivation.  Taken  together,  it  is  reasonable  to  expect  that  these  four 
features  of  intrinsic  motivation  should  be  treated  as  areas  of  concern  when  creating  a  training 
game  and  not  as  strict  requirements  until  further  research  is  completed. 
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Conclusions 


In  the  current  research,  features  of  a  PC-based  training  game  were  assessed  in  an  attempt 
to  identify  aspects  of  a  first-person-perspective  game  that  would  influence  both  the  learning  of 
content  and  player  motivation  to  continue  using  the  game.  For  the  current  research,  the  findings 
related  to  the  instructional  characteristics  (information  type,  relevance  of  information,  and 
presentation  modality)  may  be  limited  to  the  PC-based  first-person-perspective  game  used. 
However,  the  findings  mirrored  previous  findings  from  research  using  interactive  multimedia 
instruction.  Likewise,  the  findings  regarding  motivation  in  this  game  confirmed  previous 
research  on  motivational  aspects  of  other  types  of  games. 

The  assessment  of  instructional  features  suggest  that  PC-based  training  games  would  be . 
more  effective  for  learning  procedures  than  for  learning  facts.  Additional  research  would  be 
needed  to  determine  if  types  of  skills  and  procedures  influence  the  effectiveness  of  using  games 
for  training,  or  if  the  type  of  game  influences  the  effectiveness  of  the  training. 

The  findings  also  suggest  that  instructional  objectives  should  be  integrated  into  the 
game’s  storyline  so  that  the  training  material  is  relevant  to  the  progression  of  the  game.  If  the 
training  objectives  are  part  of  the  storyline  of  the  game,  the  training  effectiveness  of  the  game 
may  be  increased.  If  the  training  objectives  are  not  part  of  the  storyline,  then  players/students 
may  only  learn  how  to  play  the  game  and  not  the  training  objectives.  The  relevance  of  the 
training  objectives  also  overlaps  with  the  motivational  feature  of  realism,  which  should  increase 
the  likelihood  of  skill  transfer  from  the  game  to  the  application  of  the  skill  trained. 

Spoken  text  and  visual  images  were  found  to  be  more  effective  presentation  modalities 
than  printed  text,  suggesting  that  course  developers  should  focus  on  these  modes  of  presentation. 
It  does  not  mean  that  printed  text  should  be  abandoned  completely,  only  that  it  should  be  limited 
during  game  play.  Casual  observation  of  the  participants  in  this  research  showed  that  players 
might  skip  large  portions  of  text. 

The  assessment  of  motivational  features  suggest  that  PC-based  training  games  should  be 
designed  with  attention  to  challenge,  realism,  control,  and  opportunities  for  exploration,  which 
may  make  the  learner’s  experience  more  positive  and  motivate  them  to  continue  using  the  game. 
While  the  inclusion  of  all  of  these  features  does  not  guarantee  that  users  will  play  the  game  for 
hours  on  end,  they  are  features  that  should  be  considered  while  developing  a  training  game. 

For  a  training  game  to  be  effective,  it  should  be  both  instructional  and  motivational. 
Additionally,  some  of  the  instructional  features  identified  may  influence  motivation,  and  some  of 
the  motivational  features  identified  may  influence  instruction.  More  research  is  needed  to 
develop  a  better  understanding  of  the  interaction  of  instructional  and  motivational  influence  on 
the  training  effectiveness  of  first-person-perspective  games. 
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1 .  What  is  your  age? 
_ years 


Appendix  A 
Pre-Game  Questionnaire 

Subject  ID: 

Participant  Questions 


2.  What  is  your  level  of  confidence  using  computers  (please  circle  one)? 

1  2  3  4  5 

Low  Average  High 


3.  On  average,  how  many  hours  per  week  do  you  use  computers  (for  example,  schoolwork,  e- 
mail,  instant  messaging,  entertainment)? 

Hours/week 


4.  In  the  past  year,  on  average,  how  many  hours  per  week  have  you  spent  pla5dng  any  type  of 
video  game  (for  example,  PC-based,  Nintendo,  Playstation,  arcade)? 

_ Hours/week 

5.  Have  you  played  the  America’s  Army  PC-based  game? 

_ Yes  _ _ No 

If  yes,  approximate  number  of  hours  played. _ 

6.  What  computer  games  and/or  video  games  do  you  prefer?  (List  as  many  titles  as  you  like) 


A-1 


Circle  the  correct  answer.  If  you  are  unsure  of  the  answer,  circle  choice  E. 

1 .  How  do  you  switch  between  firing  modes  (1  shot,  or  three  round  burst)? 

A)  R 

B)  T 

C)  Right  mouse  button 

D)  Mouse  wheel 

E)  I  don’t  know 

2.  What  else  is  the  M249  machine  gun  called? 

A)  Suppression  Fire  Weapon 

B)  Squad  Automatic  Weapon 

C)  Tactical  Action  Munitions 

D)  Tactical  Automatic  Munitions 

E)  I  don’t  know 

3.  In  the  MOUT  training,  where  do  you  NEED  to  use  the  night  vision  goggles? 

A)  For  clearing  the  first  room 

B)  In  the  tunnels 

C)  On  the  entire  course 

D)  In  the  fuselage 

E)  I  don’t  know 

4.  What  are  the  Army’s  7  core  values? 

A)  loyalty,  duty,  respect,  selfless  service,  honor,  integrity,  and  personal  courage 

B)  loyalty,  dignity,  fortitude,  honesty,  bravery,  ingenuity,  and  personal  courage 

C)  loyalty,  duty,  bravery,  honor,  dignity,  respect,  and  dignity 

D)  loyalty,  dignity,  respect,  selfless  service,  honesty,  ingenuity,  and  personal  courage 

E)  I  don’t  know 

5.  How  do  you  change  marines,  or  reload  the  gun? 

A)  Press  R 

B)  Press  C 

C)  Press  E 

D)  Press  V 

E)  I  don’t  know 

6.  What  are  the  positions  that  you  shoot  from  during  the  qualifying  round? 

A)  standing  and  crouching 

B)  stonding  and  prone 

C)  crouching  and  prone 

D)  supported  and  prone 

E)  I  don’t  know 
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7.  When  running  the  obstacle  course  for  time,  where  is  the  clock  located? 

A)  Top  right 

B)  Bottom  right 

C)  Top  left 

D)  Bottom  left 

E)  I  don’t  know 

8.  What  key  do  you  press  to  get  into  position  so  that  you  can  GET  under  the  barbed  wire  in  the 
obstacle  course? 

A)  X 

B)  Y 

C)  C 

D)  Z 

E)  I  don’t  know 

9.  What  color  are  the  targets  during  rifle  qualifying? 

A)  White 

B)  Green 

C)  Blue 

D)  Red 

I  don’t  know 
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Appendix  B 

Post-game  Questionnaire 

Subject  ID: _ ■  _ 

Circle  the  correct  answer.  If  you  are  unsure  of  the  answer,  circle  choice  E. 

(Note;  the  following  codes  were  used  to  identify  the  categories  used  for  this  research  for  each 
question;  p=  procedural,  e=episodic,  f=factual,  rl=relevant,  ir=irrelevant,  gip=graphic, 
spk=spoken  text,  pm=printed  text,  and  muMnformation  was  presented  through  multiple 
modalities.  “Pre-game”  was  used  to  identify  items  that  appeared  on  the  pre-game  questionnaire. 
These  codes  did  not  appear  on  the  questionnaire  given  to  the  participants.) 

1.  What  is  the  reason  the  Sergeant  gives  for  “cooking”  off  a  grenade  before  throwing  it?  (f,  rl, 
spk) 

A)  To  make  sure  its  not  a  dud 

B)  To  make  it  detonate 

C)  So  the  enemy  won’t  throw  it  back  at  you 

D)  So  you  won’t  hit  a  friendly 

E)  I  don’t  know 

2.  How  do  you  switch  between  firing  modes  (1  shot,  or  three  round  burst)?  (pre-game) 

A)  R 

B)  T 

C)  Right  mouse  button 

D)  Mouse  wheel 

E)  I  don’t  know 

3.  What  animal  does  the  Drill  Sergeant  use  in  his  description  of  how  to  get  through  the  tunnel 
obstacle?  (e,  ir,  spk) 

A)  a  snake 

B)  a  goose 

C)  a  chicken 

D)  a  duck 

E)  I  don’t  know 

4.  What  else  is  the  M249  machine  gun  called?  (pre-game) 

A)  Suppression  Fire  Weapon 

B)  Squad  Automatic  Weapon 

C)  Tactical  Action  Munitions 

D)  Tactical  Automatic  Munitions 

E)  I  don’t  know 
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5.  During  basic  rifle  marksmanship  qualifying,  how  many  rounds  are  in  a  magazine?  (f,  rl,  mul) 

A)  10 

B)  15 

C)  20 

D)  25 

E)  I  don’t  know 

6.  What  are  the  two  types  of  obstacle  courses?  (f,  ir,pm) 

A)  Conditioning  and  Teamwork 

B)  Teamwork  and  Confidence 

C)  Conditioning  and  Confidence 

D)  Conditioning  and  Shampooing 

E)  I  don’t  know 

7.  What  is  written  on  the  lane  posts  of  the  obstacle  course?  (e,  ir,  pm) 

A)  USA 

B)  Army  of  One 

C)  Go  Army 

D)  The  Army’s  seven  core  values 

E)  I  don’t  know ' 

8.  What  other  specialized  schools  are  housed  at  Fort  Benning?  (f,  ir,  pm) 

A)  sniper,  airborne,  pathfinder,  and  ranger  schools 

B)  special  forces,  intelligence,  and  transportation  schools 

C)  artillery,  intelligence,  and  sniper  schools 

C)  ranger,  sniper,  and  signal  schools 

D)  I  don’t  know 

9.  During  basic  rifle  marksmanship  qualifying,  how  many  targets  must  you  hit  to  qualify  as  a 
marksman,  so  that  you  may  continue  with  the  game?  (f,  rl,  mul) 

A)  23 

B)  27 

C)  32 

D)  36 

E)  I  don’t  know 

1 0.  What  does  MOUT  stand  for?  (f,  ir,  pm) 

A)  Military  Options  on  Unidentified  Target 

B)  Mastering  Operations  in  Underground  Terrain 

C)  Military  Operations  in  Urban  Terrain 

D)  Mastering  Operations  and  Urban  Tactics 

E)  I  don’t  know 
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1 1 .  During  the  obstacle  course,  which  colored  lane  do  you  run  down?  (p,  rl,  spk) 

A)  Green 

B)  Blue 

C)  White 

D)  Red 

E)  I  don’t  know 

12.  In  the  MOUT  training,  where  do  you  NEED  to  use  the  night  vision  goggles?  (pre-game) 

A)  For  clearing  the  first  room 

B)  In  the  tunnels 

C)  On  the  entire  course 

D)  In  the  fuselage 

E)  I  don’t  know 

13.  What  are  the  Army’s  7  core  values?  (pre-game) 

A)  loyalty,  duty,  respect,  selfless  service,  honor,  integrity,  and  personal  courage 

B)  loyalty,  dignity,  fortitude,  honesty,  bravery,  ingenuity,  and  personal  courage 

C)  loyalty,  duty,  bravery,  honor,  dignity,  respect,  and  dignity 

D)  loyalty,  dignity,  respect,  selfless  service,  honesty,  ingenuity,  and  personal  courage 

E)  I  don’t  know 

14.  How  do  you  change  magazines,  or  reload  the  gun?  (pre-game) 

A)  Press  R 

B)  Press  C 

C)  Press  E 

D)  Press  V 

E)  I  don’t  know 

15.  During  both  rifle  qualifying  and  weapons  familiarization  what  compass  direction  do  you 
shoot?  (e,  ir,  pm) 

A)  North 

B)  South 

C)  East 

D)  West 

E)  I  don’t  know 
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16.  What  is  the  order  ofthe  obstacles  in  the  course?  (e,  ir,  grp) 

A)  side  step,  high/low  logs,  climb  up  the  ladder  to  high  beam,  wall,  log  ladder  to  the 
cargo  net,  tunnel,  low  beam  over  the  sand  pit,  and  under  the  barb  wire. 

B)  side  step,  wall,  log  ladder  to  the  cargo  net,  high/low  logs,  climb  up  the  ladder  to  high 
beam,  tunnel,  low  beam  over  the  sand  pit,  and  under  the  barb  wire. 

C)  side  step,  wall,  log  ladder  to  the  cargo  net,  high/low  logs,  climb  up  the  ladder  to  high 
beam,  tunnel,  low  beam  over  the  sand  pit,  beams  with  the  cargo  net,  and  under  the 
barb  wire. 

D)  side  step,  high/low  logs,  tunnel,  low  beam  over  the  sand  pit,  climb  up  the  ladder  to 
high  beam,  wall,  log  ladder  to  the  cargo  net,  and  under  the  barb  wire. 

E)  I  don’t  know 

17.  What  are  the  positions  that  you  shoot  from  during  the  qualifying  round?  (pre-game) 

A)  standing  and  crouching 

B)  standing  and  prone 

C)  crouching  and  prone 

D)  supported  and  prone 

E)  I  don’t  know 

18.  How  much  time  do  you  have  to  complete  the  obstacle  course?  (f,  rl,  pm) 

A)  60  seconds 

B)  90  seconds  . 

C)  120  seconds 

D)  180  seconds 

E)  I  don’t  know 

1 9.  During  MOUT  where  do  you  have  to  throw  a  flash  bang?  (p,  rl,  grp) 

A)  Down  the  tunnel 

B)  Up  the  tunnel 

C)  Into  the  first  room 

D)  Into  the  last  room 

E)  I  don’t  know 

20.  How  many  rounds  are  in  a  belt  for  the  M249  machine  gun?  (f,  rl,  spk) 

A)  20 

B)  100 

C)  200 

D)  400 

E)  I  don’t  know 

21 .  What  generally  is  the  shape  ofthe  smoke  grenade?  (e,  ir,  grp) 

A)  a  ball 

B)  a  can 

C)  a  bottle 

D)  a  box 

E)  I  don’t  know 
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22.  What  “fire  point”  does  the  drill  sergeant  send  you  to  for  rifle  training  and  qualification?  (f, 
rl,  mul) 

A)  3 

B) 5 

C) 7 

D) 9 

E)  I  don’t  know 

23.  How  do  you  bring  the  gun  up  so  that  you  can  look  directly  through  the  sites?  (p,  rl,  pm) 

A)  Press  Z 

B)  Press  A 

C)  Press  Q 

D)  Press  R 

E)  I  don’t  know 

24.  When  mnning  the  obstacle  course  for  time,  where  is  the  clock  located?  (pre-game) 

A)  Top  right 

B)  Bottom  right 

C)  Top  left 

D)  Bottom  left 

E)  I  don’t  know 

25.  What  key  do  you  press  to  get  into  position  so  that  you  can  get  under  the  barbed  wire  in  the 
obstacle  course?  (pre-game) 

A)  X 

B)  Y 

C)  C 

D)  Z 

E)  I  don’t  know 

26.  What  color  are  the  targets  during  rifle  qualifying?  (pre-game) 

A)  White 

B)  Green 

C)  Blue 

D)  Red 

E)  I  don’t  know 

27.  How  do  you  toggle  (switch)  between  sprint  and  walk?  (p,  rl,  pm) 

A)  The  Shift  button 

B)  The  Backspace  button 

C)  The  Control  button 

D)  The  Caps  Lock  button 

E)  I  don’t  know 
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28.  How  many  targets  do  you  get  a  chance  to  hit  during  your  rifle  marksmanship  qualification? 
(e,rl,grp) 

A)  20 

B)  30 

C)  40 

D)  50 

E)  I  don’t  know 

29.  When  you  are  looking  down  the  gun  sites  and  pull  the  trigger,  what  direction  does  the  aiming 
point  of  the  gun  move  (kick)?  (e,  rl,  grp) 

A) Up 

B)  Up  and  Right 

C)  Up  and  Left 

D)  Left 

E)  I  don’t  know 

30.  What  gun  do  you  use  during  your  basic  rifle  marksmanship  training  and  qualification?  (f,  ir, 
mul) 

A) M60 

B) AR-15 

C) M16A2 

D)  AR-249 

E)  I  don’t  know 

3 1 .  When  looking  down  the  gun  sites,  what  direction  does  the  gun  primarily  sway  in 
coordination  with  your  breathing?  (e,  rl,  gip) 

A)  primarily  up  then  down 

B)  primarily  up  and  right,  then  down  and  left 

C)  primarily  up  and  left,  then  down  and  right 

D)  primarily  left  then  right 

E)  I  don’t  know 

32.  What  is  the  nickname  for  Fort  Benning?  (f,  ir,  pm) 

A)  “Home  of  the  Brave” 

B)  “Home  of  the  Infantry” 

C)  “Land  of  the  Courageous” 

D)  “Where  Freedom  Rings  Trae” 

E)  I  don’t  know 

33.  Diuing  basic  rifle  marksmanship  qualifying,  how  many  targets  must  you  hit  to  qualify  to 
attend  sniper  school?  (f,  rl,  pm) 

A)  23 

B)  27 

C) 32 

D)  36 

E)  I  don’t  know 
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34.  At  the  start  of  basic  rifle  marksmanship  qualification,  where  do  you  go  to  get  your  gun  and 
ammo?  (p,  rl,  mul) 

A)  Front  and  left 

B)  Front  and  right 

C)  Back  and  left 

D)  Back  and  right 

E)  I  don’t  know 

35.  In  the  MOUT  training,  how  do  you  select  the  “Flash/Bang”  grenade?  (p,  rl,  pm) 

A)  Press  3 

B)  Press  4 

C)  Press  5 

D)  Click  Right  mouse 

E)  I  don’t  know 
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Instructions:  Please  be  specific  when  answering  the  questions. 

1 .  What  about  the  “America’s  Army”  game  would  make  you  want  to  play  the  game  again? 


2.  Which  section  of  the  basic  combat  training  did  you  like  the  most,  and  why? 


3.  Which  section  of  the  basic  combat  training  did  you  like  the  least,  and  why? 


4.  What  did  you  learn  from  the  “America’s  Army”  game? 


5.  What  would  you  change  about  the  “America’s  Army”  game 


B-8 


